Skip to content

feat: Kyverno-based cc_init_data injection for OSC 1.12#77

Open
butler54 wants to merge 10 commits intovalidatedpatterns:mainfrom
butler54:feature/kyverno-initdata
Open

feat: Kyverno-based cc_init_data injection for OSC 1.12#77
butler54 wants to merge 10 commits intovalidatedpatterns:mainfrom
butler54:feature/kyverno-initdata

Conversation

@butler54
Copy link
Copy Markdown
Collaborator

Summary

  • Add Kyverno MutatingPolicy to inject cc_init_data into pods with kata runtime classes, replacing broken io.katacontainers.config.agent.policy annotation
  • Add ValidatingPolicy to audit initdata ConfigMaps for required fields
  • Add ClusterPolicy to propagate initdata ConfigMaps from imperative namespace to workload namespaces
  • Update imperative job to generate both default and debug initdata ConfigMaps with Kyverno validation fields
  • Update values-simple.yaml: OSC 1.12.0, Trustee 1.1.0, Kyverno Helm chart, OpenShift SCC overrides
  • Trustee app temporarily targets feature/trustee-1.1-compat branch (revert after trustee-chart PR merged)

Three initdata modes

Mode Trigger ConfigMap
Default coco.io/initdata-configmap: initdata Generated by imperative job
Debug coco.io/initdata-configmap: debug-initdata Generated by imperative job (permissive policy, same KBS cert)
User-provided Pod already has cc_init_data annotation User-managed (no mutation)

Dependencies

Test plan

  • Deploy with main.clusterGroupName: simple on Azure
  • Verify Kyverno pods running in kyverno namespace
  • Verify Kyverno policies created: oc get mpol,vpol,cpol
  • Wait for imperative job → both initdata and debug-initdata ConfigMaps created
  • Verify ConfigMaps propagated to hello-openshift and kbs-access namespaces
  • Create test pod with kata runtime → verify io.katacontainers.config.hypervisor.cc_init_data injected
  • Test debug mode: pod with coco.io/initdata-configmap: debug-initdata
  • Test user-provided: pod with explicit cc_init_data, no coco.io annotation → no mutation
  • Revert trustee app to chart: trustee + chartVersion: 0.2.* after trustee-chart release

🤖 Generated with Claude Code

butler54 and others added 2 commits April 20, 2026 21:19
…e 1.1

Replace broken io.katacontainers.config.agent.policy annotation with
Kyverno MutatingPolicy that injects cc_init_data from ConfigMaps into
pods with kata runtime classes.

- Add coco-kyverno-policies chart with MutatingPolicy, ValidatingPolicy,
  and ClusterPolicy for ConfigMap namespace propagation
- Update imperative job to generate both default and debug initdata
  ConfigMaps with Kyverno validation fields
- Update workload pod templates to use coco.io/initdata-configmap
  annotation instead of inline policy
- Update values-simple.yaml: OSC 1.12, Trustee 1.1, Kyverno Helm app
- Add conditional memory annotation for non-Azure platforms
- Delete insecure-policy.rego (policy now embedded in cc_init_data)

BREAKING CHANGE: Requires Kyverno and OSC 1.12 / Trustee 1.1.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Temporarily targets butler54/trustee-chart feature/trustee-1.1-compat
branch instead of released chart version. Revert to chart: trustee +
chartVersion: 0.2.* after trustee-chart PR is merged and released.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@butler54 butler54 requested a review from a team April 20, 2026 12:21
butler54 and others added 8 commits April 21, 2026 10:05
Cloud platforms (Azure/AWS) use "kata" runtime class for peer-pods while
baremetal uses "kata-cc" for confidential containers. Updates Kyverno
MutatingPolicy to explicitly match both classes instead of startsWith,
and adds ServerSideApply syncPolicy for Kyverno CRDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ACM owns the policyreports.wgpolicyk8s.io CRD with v1beta1 as a stored
version. Kyverno's version drops v1beta1, causing a storage migration
error. Disable Kyverno's policyreport CRD installation since ACM
already provides it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… CRDs

The Kyverno chart validates that reportsController and policyreport CRDs
are consistent. Since ACM owns the policyreport CRDs and we disabled
their installation, we must also disable the reportsController. Mutation
and validation policies still work without it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace bare Pod templates with Deployments (replicas: 1) in
hello-openshift and kbs-access charts. Update the Kyverno MutatingPolicy
to target Deployments instead of Pods since the policies.kyverno.io/v1
API does not support autogen and confidential pods can't be easily
mutated. Add generateExisting to namespace propagation ClusterPolicies
and enable ArgoCD pruning to clean up old Pod resources.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bare Pods ran under kube:admin with anyuid SCC, but Deployment-created
pods use the default ServiceAccount which only has restricted-v2 SCC.
Remove the fixed runAsUser: 1001 and let OpenShift assign a UID from
the namespace range. runAsNonRoot: true ensures non-root execution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The policies.kyverno.io/v1 MutatingPolicy API doesn't support ConfigMap
lookups (namespaceObject.get is not available). Switch to kyverno.io/v1
ClusterPolicy which supports configMap context lookups and has autogen
for Deployments/StatefulSets/DaemonSets/Jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The kata runtime class runs a local QEMU VM without CDH. Cloud
peer-pods require kata-remote to spawn a VM in Azure/AWS with CDH
available at 127.0.0.1:8006. Also add kata-remote to the Kyverno
ClusterPolicy match list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chris Butler <chris.butler@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant